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ABSTRACT 

This paper offers guidelines in the consideration of 
an open or closed classroom-observation system. Factors in the 
definition of an observation approach include target, timing, focus, 
topic of observation, and data-collection system. Because there are 
advantages and drawbacks to any classroom-observation system, the 
following factors should be considered when deciding whether to use 
an open or a closed observation system: the purposes of the 
observation, the amount of time and staff available to collect and 
analyze the data, and the extent to which the behaviors and event 
under study are clearly defined. Whatever observation approach is 
used, evaluators other than the researcher should be able to 
replicate the lesson, teaching context, and sequence of activities. 
Analysis, evaluation, and utilization of data should be conducted 
after the observation. The qualitative data provided by an open-ended 
approach can be more helpful in understanding and justifying 
decisions after the observation is conducted. One table is included. 
(LMI) 
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Approaches to Classroom Observations: 
Open versus Closed Systems 

Judy Hirabayashi and Patricia Wheeler 
EREAPA Associates 
Livermore, California 



Classroom observations are frequently used for collecting data and information about 
teachers that can be used for such purposes as personnel decisions, performance 
assessment, professional development, evaluation of instructional programs, and research on 
teaching. 

There are a number of ways in which classroom observations can be done, depending upon 
the purpose(s) for the observations. While certain approaches tend to be more appropriate 
for some purposes than others, the critical concern is to obtain the quantity and quality of 
information needed for the specific purpose. Some of the options are described below and 
the tradeoffs of various approaches to collecting data and information through observations 
are discussed. 



Factors that Define Observation Approaches 

Factors that define the observation approach include target, timing, focus, and data collection 
format (see Table 1). Three of these factors define the sample of data that will be collected: 
who is observed, when the data are collected, and on what aspects of teaching data are 
collected The fourth specifies how the data are recorded. 

Target Most classroom observations focus on the teacher. However, depending on the 
purp>ose of the observation, the observer may also look at students-the entire class, a 
randomly selected group of students, a group of students with specific characteristics, or a 
single student. In addition, the observer may collect data on others in the classroom, such as 
an instructional aide, tutor, student teacher, or parent volunteer. 

Timing. Data can be collected using various timing schemes: duration, interval, and 
continuous. Timing can be for short durations where the teacher is observed for, say, five 
minutes and then the observer makes notes for five minutes; this process is repeated several 
times. In the case of interval timing, the observer makes a record of what is happening at a 
given moment at specific time intervals; for example, every three minutes the observer 
records what the teacher is doing. A third timing approach is continuous. In this case, the 
observer records data throughout the full period of the observation, usually for an entire 
lesson or activity, or for a class period. 

Focus . The focus defines the observation's domains and elements-those aspects of 
tcacliing behaviors, classroom events, and student behaviors on which data are collected. 
For example, domains could be classroom management, communication skills, knowledge 
of content, and pedagogical skills. Within each domain, the observation could cover various 
elements. In classroom management, these elements might include student grouping, 
student engagement, coordination with the aide, and timing of activities. The domains and 
elements for which data are to be collected should be determined by the purpose of the 
observations and should be solidly based on appropriate foundations (see Wheeler, 1991). 
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Data collection system . Data collection can be accomplished in a variety of ways. Some are 
"closed'* systems in which the observer records data on selected aspects of teaching. 
Others are "open" systems where the observer tries to record everything that is happening. 
Some are a combination of open and closed systems. Examples of data collection methods 
that tend to be closed systems are frequency counts, rating scales, checklists, and coding 
forms to document behaviors and events. Guided-note talcing and scripting tend to be open 
systems or a combination if they focus on only ceiiain aspects of teaching. By focusing on 
selected beliaviors and activities, guided-note taking helps ensure that data are collected on 
all domains aj.id elements covered by the system. Continuous scripting (i.e., making as 
complete a chronological record as possible) provides better data on the sequencing of the 
behaviors and events, and may include data on areas not covered by the domains and 
elements. 



Considerations and Tradeoffs in Using Open and Closed Systems 

There are advantages and drawbacks to any classroom observation system; it is important to 
be aware of the likely consequences of choosing a particular approach. Factors to consider 
in deciding whether to use an "open" or a "closed" observation system include: 

- the purposes of the observation; 

- time and staff available to collect and to analyze the data; and 

- how clearly defined the behaviors and events to be observed are. 

Some typical situations in which observations might be conducted and rationales for 
choosing one approach over another are offered below. 

What is the purpose of the observation ? (n informal situations, for example, when the 
primary reason for observing is to give general feedback about a lesson to a teacher or for 
professional development of new teachers, "face validity" may be the most important 
characteristic of the format chosen. In more formal situations, such as teacher evaluation or 
personnel decision making, there is a need for comprehensive data on many aspects of a 
teacher's performance for use in making and justifying decisions. An open system in 
which classroom dialogue and behaviors are recorded in detail is more appropriate for 
reconstructing the "reality" of the lesson and the nature of the teaching performance as a 
basis for discussions with the teacher and for decision making about a teacher's 
performance. 

At the other end of the continuum, if large numbers of observations are being made as paxi 
of a research study with a focus on specific aspects of teaching or for a formal end-of- 
project evaluation for a funding agency and there will be no individual feedback to a teacher, 
one might consider a closed format such as a checklist or rating scale that focuses only on 
areas of interest Such an approach could also be used to see how well teachers are 
incorporating, into their classroom insuniction, certain techniques that were coverec in a 
recent inservice training program. 

Who will do the observation ? If more than a few observations are to be conducted, it is 
likely that more than one person will do them. Initially, it may be more difficult to train 
observers to use a closed instrument, especially one that calls for coding of behaviors and 
events. Considerable dme and extensive pilot testing may be necessary to devise a format 
that can be used reliably with acceptable accuracy and completeness by trained observers 
and that can be analyzed in an efficient and accurate manner. In comparison, guided-note 
taking and continuous scripting are relatively easy to design and observers can usually be 
trained in less time. 



The relative amount of time required for developing and training observers with open and 
closed systems reverses during the analysis phase. Data collected using closed systems can 
be analyzed quickly and accurately using quantitative ^preaches ranging from simple 
tallies to specialized software. 

In contrast, if narratives are to be ased for more than the most informal purposes, they 
should be transcribed or typed into a standard format before being analyzed individually or 
collectively. Then, the most salient information must be identified, extracted and 
summarized if sense is to be made of the wealth of data and if users are to be spared lengthy 
descriptions of tangential and repetitive behaviors and events. Such analysis can require 
special expeilise. 

What is being observed ? When it is easy to judge whether or not something is present 
(e.g., classroom is physically conducive to cooperative learning; teacher can be heard from 
the back of the room throughout the lesson), a simple closed method for recording the 
information will probably suffice. Open system are much better for capturing subtle 
information. For example, brief snatches of dialogue extracted from observations made 
over several months or years can be very instructive about how teachers' repertoires of 
questioning techniques evolve and how students are responding to new types of questions. 
Stodolsky ( 1990) feels that, . . the use of checklists of specific behaviors is compatible 
with the idea that teaching consists of certain classes of discrete teaching acts, but is not 
compatible with a view of teaching as preparing an environment for student learning" (p. 
177). Wood (1992) points out that, . . if principals focus observations on discrete 
teacher behaviors, they may tend to create tunnel vision that will impede their ability to see 
what actually occurs in the classroom. If the intent of evaluation is to help teachers improve 
their instructional practices, the categories and descriptors may become obstacles to seeing 
rather tfian aids to better vision" (p. 55). 

Open systems also allow one to collect data on '*what is" rather than on a preconceived 
notion of what will be observed. Key data can be collected, even in areas not covered by the 
foundations and domains underiying the observations, Milner ( 1991) says that, "An 
'objective' instrument cannot measure the 'subjective' dimensions of a classroom-that 
dimension which gives teachik^g its creative force" (p. 464). 

An open system can be advantageous when the domains and elements for an observation 
may need to change as behaviors change over time. For example, early in a project to 
restructure math education or in the evaluation of new math teachers, one might focus 
heavily on the math content being presented and instructional modes. SeverS months or 
years later, observations might need to focus on how the teacher uses writing in math and 
otherwise integrates math with other subjects. In fast-changing areas such as math and 
science instruction in California in the 1980s, one could not necessarily have predicted 
several years ago what would be observable today. Teachers may 'Hop out" on an 
instrument used to describe their behaviors in past years because of wide-spread adoption 
of or training in certain teaching behaviors. In addition, some instructional strategies or 
teaching responsibilities covered by the observation data collection instrument may no 
longer be used or applicable. 

Often the ideal solution can be to collect some information in both open and closed formats, 
with one format enriching the other. It is easier to make sense of narrative reports, for 
example, if the observer has made a simultaneous record of changes in insuiictional mode 
or student grouping. Quotes of classroom dialogue can justify judgments made on rating 
scales and later be used to illustrate and enliven quantitative reports and evaluation 
summaries. 



Summary 



Whatever observation approach is used to collect and record data, the observer as well as 
someone other than the observer should be able to "reconstruct the lesson, including the 
sequence of instructional activities the teacher employed and . . . produce a record phrased 
in terms of specific behaviors" (National Association of Elementary School Principals, 
1988, p. 7). The data collected should be just that-data- facts, evidence, quotes, descriptive 
notes. The data should reflect the teaching context and the sequence of ?ctivities in the 
classroom as completely as possible. 

Popham (1987) describes "champagne" systems of teacher evaluation as ones "bathed in 
bubbles" on a form. He discourages the use of such closed systems, stating that, "By 
casting as much as possible of the system in a form that yields quantifiable information or, 
putting it more directly, numbers, the designers of champagne systems hope to create 
sufficient rigor in the system so that its dominantly quantitative product will be viewed with 
credulity. But numbers, unfortunately, are no protection against nonsense" (p. 28). 

Analysis, evaluation, judgments, and use of observation results should be done after the 
observation. For these processes to be done in a sound manner from technical, professional 
and legal perspectives, it is essential that the data collection approach used provide an 
accurate and a comprehensive documentation record that can be used by other individuals 
working with the documentation or trying to understand the interpretation. The qualitative 
data provided by use of an open approach can be much more helpful in later understanding 
and justifying decisions made and in helping a teacher understand his/her strengths and 
weaknesses £an a series of numbers, the bases for which are unclear or undocumented 
Guidelines for use of an open system are provided in Wheeler (1992). 
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TABLE 1 

Factors Deflning a Classroom Observation System 



Factor Definition 



Target Who is observed 



Timing When data are 

collected 



Focus Domains on which 

data are collected 



Format How data are 

recorded 



Examples 



Teacher 

Students (all, some, individuals) 
Instructional aides, tutors 
Others in classroom 

Duration 

Interval 

Continuous 

Knowledge of content 
Pedagogical skills 
Assessment techniques 
Classroom management 
Communication 
Use of resources 
CliiHate for learning 
Physical setting 

Closed: 

Checklists 

Coding forms 

Frequency counts 

Ratinp. - al^^ 
Open: 

Guided-note taking 

Scripting 
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